Parallel Processor Node
The Parallel Processor Node processes collections of items concurrently using a specified processor node with automatic load balancing and result aggregation. It distributes workload across multiple workers for improved throughput while maintaining result ordering. Processing failures are handled gracefully with configurable error handling modes.
How It Works
When the node executes, it creates isolated execution contexts for each item, ensuring that processing one item doesn't interfere with others. Each worker receives a single item, processes it through the specified processor node, and returns the result. The Parallel Processor maintains the original order of results, so the output array matches the input array's sequence regardless of which items complete first.
The node automatically detects whether the processor node is asynchronous or synchronous and uses the appropriate execution strategy. For asynchronous processors, it uses semaphore-based concurrency control to maintain the specified worker limit. For synchronous processors, it uses thread pools to achieve parallelism.
Progress tracking updates in real-time as items complete, allowing monitoring through streaming messages that show current progress, completion percentage, and item counts.
Configuration Parameters
Input field
Input Field (Text, Required): Workflow variable containing a collection of items to process.
The input must be an array or list. Each item is passed individually to the processor node. Items can be any type (objects, strings, numbers, arrays) as long as the processor node handles that data type.
Output field
Output Field (Text, Required): Workflow variable where processed results are stored.
The output is an array of results corresponding to input items, maintaining the same order. When Error Handling Mode is "Include Null", failed items appear as null values preserving indices. When "Skip Failed", failed items are removed entirely.
Common naming patterns: processed_items, parallel_results, batch_output.
Max workers
Max Workers (Number, Default: 5): Maximum concurrent workers for parallel processing (1-50).
With 100 items and 5 workers, exactly 5 items process at any time—as each completes, the next starts immediately. Setting to 1 processes items sequentially. The actual number is automatically capped at CPU count and total items.
Tune based on system resources, external API rate limits, and processor node computational intensity. Start with 5 and adjust based on performance—excessive parallelism can overwhelm APIs or exhaust resources.
Error handling mode
Error Handling Mode (Dropdown, Default: Include Null): How to handle failed items.
| Mode | Output behavior | Use when |
|---|---|---|
| Include Null (preserve indices) | Failed items appear as null at original positions | Array indices needed for downstream processing or identifying failures by position |
| Skip Failed (compact results) | Failed items removed entirely | Only successful results matter, position tracking not needed |
Failed items are logged with error details regardless of mode. Processing continues for remaining items when individual items fail.
Current item variable
Current Item Variable (Text, Optional): Variable to store current item being processed.
Creates a scoped variable accessible to the processor node, automatically populated with each item before execution. Each parallel worker has its own isolated copy, preventing race conditions. Leave empty for auto-generated name, or specify custom name for clarity when debugging.
Common parameters
This node supports common parameters shared across workflow nodes, including Stream Output Response, Streaming Messages, Logging Mode, and Wait For All Edges. For detailed information, see Common Parameters.
The Streaming Messages parameter includes an additional "Progress Message" field specific to this node for real-time progress updates. Use template variables ${current}, ${total}, and ${percentage} for dynamic progress information.
Best practices
- Set Max Workers based on bottleneck: lower values (2-5) for rate-limited APIs, higher values (10-20) for CPU-intensive operations
- Choose "Include Null" when downstream nodes need to correlate results by index position; "Skip Failed" when only successful results matter
- Structure processor nodes to handle items independently without shared state, as parallel execution means items complete in any order
- Use Current Item Variable to pass data cleanly rather than relying on global workflow variables that could cause race conditions
- Test with a small batch first to verify correct behavior before scaling to large collections
Limitations
- Processor node selection: The processor node must be configured before adding the Parallel Processor node. The UI node selector requires an existing node to reference.
- State isolation: The processor node receives a copy of workflow state with the current item variable set. Changes to other workflow variables within the processor node are not propagated back—only the output field value is captured.
- Timeout scope: Timeout configuration applies to each individual item's processing time, not the entire batch. Implement total batch time limits in parent workflow logic.
- Memory consumption: Processing large collections with high worker counts can consume significant memory. Monitor usage when processing thousands of items concurrently.